Controlling electronic devices based on wireless ranging

ABSTRACT

A wireless communication device may wirelessly control an object, such as a physical device, directly or through interaction with a virtual representation (or placeholder) of the object situated at a predefined physical location. In particular, the wireless communication device may identify an intent gesture performed by a user that indicates intent to control the object. For example, the intent gesture may involve pointing or orienting the wireless communication device toward the object, with or without additional input. Then, the wireless communication device may determine the object associated with the intent gesture using wireless ranging and/or device orientation. Moreover, the wireless communication device may interpret sensor data from one or more sensors associated with the wireless communication device to determine an action gesture corresponding to a command or a command value. The wireless communication device may then transmit the command value to control the object.

CROSS-REFERENCE TO RELATED APPLICATION

The present application is a continuation of U.S. patent Ser. No.17/142,171, filed Jan. 5, 2021, entitled “CONTROLLING ELECTRONIC DEVICESBASED ON WIRELESS RANGING,” set to issue Aug. 23, 2022 as U.S. Pat. No.11,425,767, which is a continuation of U.S. application Ser. No.16/525,408, filed Jul. 29, 2019, entitled “CONTROLLING ELECTRONICDEVICES BASED ON WIRELESS RANGING,” issued Feb. 2, 2021 as U.S. Pat. No.10,912,136, which is a continuation of U.S. application Ser. No.15/424,394, filed Feb. 3, 2017, entitled “CONTROLLING ELECTRONIC DEVICESBASED ON WIRELESS RANGING,” issued Jul. 30, 2019 as U.S. Pat. No.10,368,378, which claims the benefit of U.S. Provisional Application No.62/291,504, filed on Feb. 4, 2016, entitled “CONTROLLING ELECTRONICDEVICES BASED ON WIRELESS RANGING,” the contents of all of which areincorporated by reference herein in their entirety for all purposes.

This application is related to U.S. patent application Ser. No.16/827,610, filed Mar. 23, 2020, entitled “DISPLAYING INFORMATION BASEDON WIRELESS RANGING”, which is a continuation of U.S. patent applicationSer. No. 15/424,408, filed Feb. 3, 2017, entitled “DISPLAYINGINFORMATION BASED ON WIRELESS RANGING”, issued Mar. 24, 2020 as U.S.Pat. No. 10,602,556, the contents of all of which are incorporated byreference herein in their entirety for all purposes.

FIELD

The described embodiments relate to wireless communications amongelectronic devices and user interfaces, including systems and techniquesfor controlling electronic devices using gestures and metrics, such aswireless ranging.

BACKGROUND

The usefulness and, therefore, the popularity of many electronicdevices, including portable electronic devices (such as cellulartelephones), is often gated by ease of use. In particular, the ease ofuse of many electronic devices is typically determined by the userinterface. The user interface is the gateway through which users'actions and/or behaviors are defined and received, including userattempts to access the features of an electronic device. Consequently,the user interface is integral to an overall user experience.

However, there are limitations associated with existing user interfaces,especially the user interfaces for small, handheld electronic devices.For example, many existing user interfaces are constrained by the needfor the user to activate or turn on an electronic device. Then, the usermay need to make physical contact with the user interface, such as isthe case with user interfaces displayed on touch-sensitive displays.Therefore, many existing user interfaces produce frustrating userinteractions, which can degrade the user experience.

SUMMARY

Embodiments that relate to a wireless communication electronic devicethat controls an object are disclosed. During operation, the wirelesscommunication device identifies a received intent gesture indicatingintent to control the object, where the object is located proximate tothe wireless communication device in an environment. Then, the wirelesscommunication device determines the object associated with the intentgesture using wireless ranging. Moreover, the wireless communicationdevice accesses sensor data from one or more sensors associated with thewireless communication device and interprets the accessed sensor data todetermine a command value, which is transmitted to control the object.

Note that the object may be a physical object. For example, the physicalobject may include: a computing device, a display, a printer, acommunications device, an audio device, an appliance, a wearable device,a home automation device, an environmental control, and/or an accessory.Alternatively, the object may be a virtual object. Further, the virtualobject may function as a proxy to associate the command value with aphysical object.

Moreover, the wireless ranging may use a wireless transmissioncharacteristic or wireless ranging computation to determine a distancebetween the wireless communication device and the object. Furthermore,the accessed sensor data may include an orientation value correspondingto the wireless communication device, the orientation value beingdetermined using a compass, an accelerometer, and/or a gyroscope.

Additionally, the command value may include a command to: pair thewireless communication device with the object; change an operationalsetting of the object; and/or execute a function. Note that the commandvalue may cause the object to transmit a response including a sensorvalue and/or may request access to a resource.

In some embodiments, the wireless communication device can determine theproximity of the object in the environment using an ultrasonic chirp.

Other embodiments provide a computer-program product for use with thewireless communication device. This computer-program product includesinstructions for at least some of the operations performed by thewireless communication device.

Still other embodiments provide a method that includes one or more ofthe operations performed by the wireless communication device.

This Summary is provided for purposes of illustrating some exemplaryembodiments, so as to provide a basic understanding of some aspects ofthe subject matter described herein. Accordingly, it will be appreciatedthat the above-described features are only examples and should not beconstrued to narrow the scope or spirit of the subject matter describedherein in any way. Other features, aspects, and advantages of thesubject matter described herein will become apparent from the followingDetailed Description, Figures, and Claims.

BRIEF DESCRIPTION OF THE DRAWINGS

The included drawings are for illustrative purposes and serve only toprovide examples of possible structures and arrangements for thedisclosed systems and techniques for intelligently and efficientlymanaging operation, control, interaction and other communicationsbetween multiple associated devices. These drawings in no way limit anychanges in form and detail that may be made to the embodiments by oneskilled in the art without departing from the spirit and scope of theembodiments. The embodiments will be readily understood by the followingdetailed description in conjunction with the accompanying drawings,where like reference numerals designate like structural elements.

FIG. 1 is a block diagram illustrating an example of electronic devicescommunicating wirelessly.

FIG. 2 provides a block diagram illustrating an example of a wirelesscommunication device that performs wireless ranging.

FIGS. 3, 4, and 5 provide drawings illustrating examples of control ofan object using one or more gestures based on wireless ranging.

FIG. 6 provides an example method for controlling an object using awireless communication device.

FIG. 7 provides an example of communication among the electronic devicesin FIG. 1 .

Note that like reference numerals refer to corresponding partsthroughout the drawings. Moreover, multiple instances of the same partare designated by a common prefix separated from an instance number by adash.

DETAILED DESCRIPTION

The disclosed embodiments relate to a wireless communication device(such as a smartphone or a smart watch) that uses one or moremeasurements (such as wireless ranging or radio-based distancemeasurements) to remotely control an object, such as a physical deviceor a virtual representation (or placeholder) associated with a physicaldevice, at a physical reference location. In particular, the wirelesscommunication device may implement a user-interface technique in whichone or more measurements are used to remotely (e.g., from a distanceand, in general, without physical contact) control operation of theobject. Note that the object is sometimes referred to as a ‘controllabledevice’ or a ‘target device.’

This user-interface technique may remove the constraints associated withmany existing user interfaces. For example, a user may no longer need toopen or unlock the wireless communication device (e.g., by providing apasscode or a biometric identifier, such as a fingerprint) in ordercontrol the object. Similarly, by facilitating control of the objectfrom a distance, this user-interface technique may eliminate the needfor a user to be in physical contact with a user interface on orassociated with the object. Consequently, the user-interface techniquemay improve the user experience when using the wireless communicationdevice, and thus may increase customer satisfaction and retention.

Note that the communication used during wireless communication betweenelectronic devices in the user-interface technique may be in accordancewith a communication protocol, such as: an Institute of Electrical andElectronics Engineers (IEEE) 802.11 standard (which is sometimesreferred to as Wi-Fi). For example, the communication may be used withIEEE 802.11ax, which is used as an illustrative example in thediscussion that follows. However, the user-interface technique may alsobe used with a wide variety of other communication protocols, and inelectronic devices (such as portable electronic devices or mobiledevices) that can incorporate multiple different radio accesstechnologies (RATs) to provide connections through different wirelessnetworks that offer different services and/or capabilities.

In particular, the wireless communication device can include hardwareand software to support a wireless personal area network (WPAN)according to a WPAN communication protocol, such as those standardizedby the Bluetooth® Special Interest Group (in Kirkland, Wash.) and/orthose developed by Apple (in Cupertino, Calif.) that are referred to asan Apple Wireless Direct Link (AWDL). Moreover, the wirelesscommunication device can communicate via: a wireless wide area network(WWAN), a wireless metro area network (WMAN) a WLAN, near-fieldcommunication (NFC), a cellular-telephone or data network (such as usinga third generation (3G) communication protocol, a fourth generation (4G)communication protocol, e.g., Long Term Evolution or LTE, LTE Advanced(LTE-A), a fifth generation (5G) communication protocol, or otherpresent or future developed advanced cellular communication protocol)and/or another communication protocol.

The wireless communication device, in some embodiments, can also operateas part of a wireless communication system, which can include a set ofclient devices, which can also be referred to as stations, clientelectronic devices, or client electronic devices, interconnected to anaccess point, e.g., as part of a WLAN, and/or to each other, e.g., aspart of a WPAN and/or an ‘ad hoc’ wireless network, such as a Wi-Fidirect connection. In some embodiments, the client device can be anyelectronic device that is capable of communicating via a WLANtechnology, e.g., in accordance with a WLAN communication protocol.Furthermore, in some embodiments, the WLAN technology can include aWi-Fi (or more generically a WLAN) wireless communication subsystem orradio, and the Wi-Fi radio can implement an Institute of Electrical andElectronics Engineers (IEEE) 802.11 technology, such as one or more of:IEEE 802.11a; IEEE 802.11b; IEEE 802.11g; IEEE 802.11-2007; IEEE802.11n; IEEE 802.11-2012; IEEE 802.11ac; IEEE 802.11ax, or otherpresent or future developed IEEE 802.11 technologies.

Thus, in some embodiments, the wireless communication device can act asa communications hub that provides access to a WLAN and/or to a WWANand, thus, to a wide variety of services that can be supported byvarious applications executing on the wireless communication device.Thus, the wireless communication device may include an ‘access point’that communicates wirelessly with other electronic devices (such asusing Wi-Fi), and that provides access to another network (such as theInternet) via IEEE 802.3 (which is sometimes referred to as ‘Ethernet’).

Additionally, it should be understood that the electronic devicesdescribed herein may be configured as multi-mode wireless communicationdevices that are also capable of communicating via different 3G and/orsecond generation (2G) RATs. In these scenarios, a multi-mode electronicdevice or UE can be configured to prefer attachment to LTE networksoffering faster data rate throughput, as compared to other 3G legacynetworks offering lower data rate throughputs. For example, in someimplementations, a multi-mode electronic device is configured to fallback to a 3G legacy network, e.g., an Evolved High Speed Packet Access(HSPA+) network or a Code Division Multiple Access (CDMA) 2000Evolution-Data Only (EV-DO) network, when LTE and LTE-A networks areotherwise unavailable.

Wireless ranging can be performed using any standard or proprietaryranging technique, or any combination of standard and/or proprietaryranging techniques. A wireless ranging operation can be performed todetermine a distance between devices (e.g., between an initiator and aresponder), a direction between devices, or both. For example, a Time ofFlight/Time of Arrival (ToF/ToA) can be determined for one or moremessages between the devices, which can be used to establish a measureof distance. The one or more messages can have any format and can betransmitted using any wireless protocol, e.g., an 802.11 protocol,Bluetooth, etc. In some embodiments, ToF/ToA can be determined using abi-directional exchange of two or more messages. Also, in someembodiments, one or more messages used to perform ranging can besecured, e.g., by encrypting or otherwise protecting at least a portionof the content. Further, in some embodiments, the direction of thesource of one or more wireless signals can be determined using atechnique such as Angle of Arrival (AoA). For example, AoA estimationcan be performed using multiple receive elements (e.g., elements of anantenna array) to measure the different times (TDOA) and/or differentphases (PDOA) of arrival of a signal. Additionally or alternatively, insome embodiments, directionality can be determined by measuring Dopplershifts to establish a frequency difference of arrival (FDOA). Wirelessranging techniques can be applied individually or in combination toperform a single ranging operation. Further, wireless ranging techniquescan be applied individually or in combination to perform on-goingranging operations, such as continuous or intermittent ranging, and ahistory of measurements can be captured and used in performingoperations based on range and/or direction.

In accordance with various embodiments described herein, the terms‘wireless communication device,’ ‘electronic device,’ ‘mobile device,’‘mobile station,’ ‘wireless station,’ ‘wireless access point,’‘station,’ ‘access point’ and ‘user equipment’ (UE) may be used hereinto describe one or more consumer electronic devices that may be capableof performing procedures associated with various embodiments of thedisclosure.

We now describe embodiments of the user-interface technique. FIG. 1presents a block diagram 100 illustrating an example of electronicdevices communicating wirelessly. In particular, a wirelesscommunication device 110 (such as a smartphone, a laptop computer, awearable, or a tablet) and physical device 112 may communicatewirelessly. These electronic devices may wirelessly communicate while:detecting one another by scanning wireless channels, transmitting andreceiving beacons or beacon frames on wireless channels, establishingconnections (for example, by transmitting connect requests), and/ortransmitting and receiving packets or frames (which may include therequest and/or additional information, such as data, as payloads).Further, there may be a virtual representation 114 at a physicallocation 116. The virtual representation 114 can correspond to aphysical device 118, such that virtual representation 114 functions as aproxy for physical device 118. In this case, wireless communicationdevice 110 may identify and interact with virtual representation 114,but may transmit wireless signals that are received at physical device118. For example, virtual representation 114 may be associated with athermostat and an adjustment by wireless communication device 110 of thethermostat may be provided through interaction with virtualrepresentation 114, but received and implemented by an environmentalunit, e.g., physical device 118.

Note that physical devices 112 and/or 118 may include: an appliance(such as an oven, a toaster, a refrigerator, a dish washer or a laundrymachine), another electronic device (such as a computer, a laptop, atablet or a computing device), an entertainment device (such as atelevision, a display, a radio receiver or a set-top box), an audiodevice, a projector, a security device (such as an alarm or a doorlock), a communication device (such as a smartphone), a monitoringdevice (such as a smoke detector or a carbon-monoxide detector), anenvironmental control (such as a thermostat, a light switch, or ashade), an accessory (such as a keyboard, a mouse or a speaker), aprinter, a wearable device, a home-automation device, a resource in anenvironment 108 (such as a transportation resource, a shared computingresource, a medical resource, a display resource, a security resource,an accessibility resource or a safety resource), etc. Moreover, virtualrepresentation 114 may be implemented as: a sticker, a picture, a pieceof ceramic, a geo-fence, one or more coordinates defining a location,etc. In some embodiments, physical device 118 includes: a light switch,a thermostat, etc.

As described further below with reference to FIG. 2 , wirelesscommunication device 110, physical device 112, and/or physical device118 may include subsystems, such as a networking subsystem, a memorysubsystem and a processor subsystem. In addition, wireless communicationdevice 110, physical device 112, and/or physical device 118 may includeradios 120 in the networking subsystems. More generally, wirelesscommunication device 110, physical device 112, and/or physical device118 can include (or can be included within) any electronic devices withnetworking subsystems that enable wireless communication device 110,physical device 112, and/or physical device 118 to wirelesslycommunicate with another electronic device. This can includetransmitting beacons on wireless channels to enable electronic devicesto make initial contact with or to detect each other, followed byexchanging subsequent data/management frames (such as connect requests)to establish a connection (which is sometimes referred to as a ‘Wi-Ficonnection’), configure security options (e.g., IPSec), transmit andreceive packets or frames via the connection, etc.

As can be seen in FIG. 1 , wireless signals 122 (represented by a jaggedline) are communicated by radios 120-1 and 120-2 in wirelesscommunication device 110 and physical device 112, respectively. Forexample, wireless communication device 110 and physical device 112 mayexchange packets using a Bluetooth protocol in a wireless personal areanetwork (WPAN) or a Wi-Fi protocol in a wireless local area network(WLAN).

In particular, as described further below with reference to FIGS. 3-5 ,wireless communication device 110 may transmit a frame or a packet thatincludes a transmission time. When this frame or packet is received byphysical device 112, the arrival time may be determined. Based on theproduct of the time of flight (the difference of the arrival time andthe transmission time) and the speed of propagation, the distancebetween wireless communication device 110 and physical device 112 can becalculated. This distance may be communicated in a subsequenttransmission of a frame or a packet from physical device 112 to wirelesscommunication device 110 along with an identifier (such as a uniqueidentifier) of physical device 112 or a user of physical device 112.Alternatively, physical device 112 may transmit a frame or a packet thatincludes a transmission time and an identifier of physical device 112,and wireless communication device 110 may determine the distance betweenwireless communication device 110 and physical device 112 based on theproduct of the time of flight (the difference of a arrival time and thetransmission time) and the speed of propagation. Note that this approachfor dynamically determining distances between electronic devices thatwirelessly communicate is sometimes referred to as ‘wireless ranging.’Further, wireless ranging (separately or along with other sensor input,such as a compass, gyroscope and/or accelerometer) can be used todisambiguate control input intent when multiple target devices may belocated close to one another or in the same line of sight. A variationon this approach may be used, in which wireless communication device 110senses gesture input directed at physical location 116 through sensorinput (e.g., compass, gyroscope and/or accelerometer) and determinesthat one or more control signals should be transmitted to an associateddevice, e.g., physical device 118 associated with virtual representation114. Similarly, another variation on this approach in which wirelesscommunication device 110 transmits frames or packets that are reflectedat physical location 116 may optionally be used to dynamically determinethe distance between wireless communication device 110 and virtualrepresentation 114. Thus, wireless ranging may be used by wirelesscommunication device 110 to determine when an object (such as physicaldevice 112 or virtual representation 114) is proximate in environment108.

While the preceding example illustrated wireless ranging withsynchronized clocks in wireless communication device 110 and physicaldevice 112, in other embodiments the clocks are not synchronized. Forexample, the position of wireless communication device 110 or physicaldevice 112 may be estimated based on the speed of propagation and thetime of arrival data of wireless signals 122 at several receivers atdifferent known locations (which is sometimes referred to as‘differential time of arrival’) even when the transmission time isunknown or unavailable. More generally, a variety of radiolocationtechniques may be used, such as: determining distance based on adifference in the power of the received signal strength indicator (RSSI)relative to the original transmitted signal strength (which may includecorrections for absorption, refraction, shadowing and/or reflection);determining the angle of arrival at a receiver (includingnon-line-of-sight reception) using a directional antenna or based on thedifferential time of arrival at an array of antennas with knownlocation(s); determining the distance based on backscattered wirelesssignals; and/or determining the angle of arrival at two receivers havingknown location (i.e., trilateration or multilateration). Note thatwireless signals 122 may include transmissions over GHz or multi-GHzbandwidths to create pulses of short duration (such as, e.g.,approximately 1 ns), which may allow the distance to be determinedwithin 0.3 m (e.g., 1 ft.). In some embodiments, the wireless ranging isfacilitated using location information, such as a location of one ormore of electronic devices in FIG. 1 that are determined or specified bya local positioning system, a Global Positioning System and/or awireless network.

Moreover, wireless communication device 110 may include one or moresensors that measure (or generate) sensor data. For example, the one ormore sensors may include: one or more compasses, one or moreaccelerometers, and/or one or more gyroscopes that measure anorientation (or an orientation value) or a direction of wirelesscommunication device 110; one or more accelerometers that measures anacceleration of wireless communication device 110; a transceiver (suchas radio 120-1) that determines a metric that characterizes wirelesscommunication between wireless communication device 110 and anotherelectronic device (such as physical device 112 or, in embodiments inwhich reflected wireless signals are received, wireless communicationdevice 110); one or more touch sensors configured to receive touchinput, e.g., via a touch screen; and/or one or more microphones oracoustic transducers that measure ambient sound in environment 108 ofwireless communication device 110. In some embodiments, wirelesscommunication device 110 determines the proximity of the object inenvironment 108 using an ultrasonic chirp provided by an acoustictransducer. This ultrasonic chirp may be outside the range of humanhearing. In the discussion that follows, ‘proximity’ of electronicdevices should be understood to include at least being withinwireless-communication range, and may further restrict the electronicdevices to be in the same room or within a predefined distance (such aswithin 10 m or 30 m).

As described further below with reference to FIGS. 3-5 , during theuser-interface technique, wireless communication device 110 may usewireless ranging and/or the sensor data to control physical devices 112and/or 118. In particular, wireless communication device 110 mayidentify an intent gesture of a user of wireless communication device110. For example, the user may point wireless communication device 110toward physical location 116 (and, thus, towards virtual representation114) or toward physical device 112, and wireless communication device110 may identify this ‘intent gesture’ based on the orientation ofwireless communication device 110 (either in isolation or relative tothe position and/or orientation of physical device 112 or virtualrepresentation 114) and/or movement of the wireless communication device110 (e.g., a predefined intent gesture motion, such as a forwardextension (or push)). Then, wireless communication device 110 maydetermine the object associated with the intent gesture (such asphysical device 112 or virtual representation 114), including usingwireless ranging. For example, the object corresponding to the intentgesture may be determined based at least in part on the distance to theobject determined using wireless ranging as well as the orientation (orrelative orientations).

Moreover, the user may then perform an ‘action gesture’ by movingwireless communication device 110, e.g., in two or three dimensions.This action gesture may be captured in the detected/measured sensor dataacquired by one or more sensors in or associated with wirelesscommunication device 110. Next, wireless communication device 110 mayinterpret the sensor data to determine the action gesture and, thus, acommand or command value. Furthermore, wireless communication device 110may transmit, in a frame or a packet, the command or command value tothe corresponding device to be controlled, e.g., physical device 112 orphysical device 118.

As described further below, note that the command value may include acommand to: pair or associate wireless communication device 110 with theobject; change an operational setting of the object (such as turninglights on or off, or change a temperature of a thermostat, adjustingvolume, channel, and/or playback settings); transfer data (such as mediaor text data); and/or execute any other such function. Note that thecommand value may cause the object to transmit a response, e.g.,including a sensor value, back to wireless communication device 110and/or may request access to a resource (such as a wireless printer).

Thus, the user-interface technique may allow wireless communicationdevice 110 to control an object (or functionality associated with anobject) from a distance, including without the user opening or unlockingwireless communication device 110. This capability may also provide theuser new degrees of freedom in controlling or interacting with theobject. Consequently, the user-interface technique may improve the userexperience when using wireless communication device 110 and the object,and thus may increase user satisfaction and retention.

In the described embodiments, processing a packet or frame in one ofwireless communication device 110, physical device 112, and/or physicaldevice 118 includes: receiving wireless signals 122 encoding the packetor frame; decoding/extracting the packet or frame from received wirelesssignals 122 to acquire the packet or frame; and processing the packet orframe to determine information contained in the packet or frame (such asdata in the payload).

In general, the communication via the WLAN in the user-interfacetechnique may be characterized by a variety of metrics (orcommunication-performance metrics). For example, the metric may include:an RSSI, a data rate, a data rate for successful communication (which issometimes referred to as a ‘throughput’), an error rate (such as a retryor resend rate), a mean-square error of equalized signals relative to anequalization target, inter-symbol interference, multipath interference,a signal-to-noise ratio, a width of an eye pattern, a ratio of number ofbytes successfully communicated during a time interval (such as 1-10 s)to an estimated maximum number of bytes that can be communicated in thetime interval (the latter of which is sometimes referred to as the‘capacity’ of a communication channel or link), and/or a ratio of anactual data rate to an estimated data rate (which is sometimes referredto as ‘utilization’).

Although we describe the network environment shown in FIG. 1 as anexample, in alternative embodiments, different numbers or types ofelectronic devices may be present. For example, some embodiments includemore or fewer electronic devices. As another example, in anotherembodiment, different electronic devices can be transmitting and/orreceiving packets or frames.

We now describe embodiments of an electronic device. FIG. 2 presents ablock diagram of an example of electronic device 200 (which may be aportable electronic device or a station). For example, electronic device200 may be one of: wireless communication device 110, physical device112, and/or physical device 118 in FIG. 1 . Electronic device 200 mayinclude processing subsystem 210, memory subsystem 212, networkingsubsystem 214, display subsystem 226, measurement subsystem 230, anduser-interaction subsystem 232. Processing subsystem 210 includes one ormore devices configured to perform computational operations. Forexample, processing subsystem 210 can include one or moremicroprocessors, application-specific integrated circuits (ASICs),microcontrollers, programmable-logic devices, and/or one or more digitalsignal processors (DSPs).

Memory subsystem 212 includes one or more devices for storing dataand/or instructions for processing subsystem 210 and networkingsubsystem 214. For example, memory subsystem 212 can include dynamicrandom access memory (DRAM), static random access memory (SRAM), aread-only memory (ROM), flash memory, and/or other types of memory. Insome embodiments, instructions for processing subsystem 210 in memorysubsystem 212 include: one or more program modules or sets ofinstructions (such as program module 222 or operating system 224), whichmay be executed by processing subsystem 210. For example, a ROM canstore programs, utilities or processes to be executed in a non-volatilemanner, and DRAM can provide volatile data storage, and may storeinstructions related to the operation of electronic device 200. Notethat the one or more computer programs may constitute a computer-programmechanism or software. Moreover, instructions in the various modules inmemory subsystem 212 may be implemented in: a high-level procedurallanguage, an object-oriented programming language, and/or in an assemblyor machine language. Furthermore, the programming language may becompiled or interpreted, e.g., configurable or configured (which may beused interchangeably in this discussion), to be executed by processingsubsystem 210. In some embodiments, the one or more computer programsare distributed over a network-coupled computer system so that the oneor more computer programs are stored and executed in a distributedmanner.

In addition, memory subsystem 212 can include mechanisms for controllingaccess to the memory. In some embodiments, memory subsystem 212 includesa memory hierarchy that includes one or more caches coupled to a memoryin electronic device 200. In some of these embodiments, one or more ofthe caches is located in processing subsystem 210.

In some embodiments, memory subsystem 212 is coupled to one or morehigh-capacity mass-storage devices (not shown). For example, memorysubsystem 212 can be coupled to a magnetic or optical drive, asolid-state drive, or another type of mass-storage device. In theseembodiments, memory subsystem 212 can be used by electronic device 200as fast-access storage for often-used data, while the mass-storagedevice is used to store less frequently used data.

Networking subsystem 214 includes one or more devices configured tocouple to and communicate on a wired and/or wireless network (i.e., toperform network operations), including: control logic 216, an interfacecircuit 218 and a set of antennas 220 (or antenna elements) in anadaptive array that can be selectively turned on and/or off by controllogic 216 to create a variety of optional antenna patterns or ‘beampatterns.’ (While FIG. 2 includes set of antennas 220, in someembodiments electronic device 200 includes one or more nodes, such asnodes 208, e.g., a pad, which can be coupled to set of antennas 220.Thus, electronic device 200 may or may not include set of antennas 220.)For example, networking subsystem 214 can include a Bluetooth™networking system, a cellular networking system (e.g., a 3G/4G networksuch as UMTS, LTE, etc.), a universal serial bus (USB) networkingsystem, a networking system based on the standards described in IEEE802.11 (e.g., a Wi-Fi® networking system), an Ethernet networkingsystem, and/or another networking system.

Networking subsystem 214 includes processors, controllers,radios/antennas, sockets/plugs, and/or other devices used for couplingto, communicating on, and handling data and events for each supportednetworking system. Note that mechanisms used for coupling to,communicating on, and handling data and events on the network for eachnetwork system are sometimes collectively referred to as a ‘networkinterface’ for the network system. Moreover, in some embodiments a‘network’ or a ‘connection’ between the electronic devices does not yetexist. Therefore, electronic device 200 may use the mechanisms innetworking subsystem 214 for performing simple wireless communicationbetween the electronic devices, e.g., transmitting advertising or beaconframes and/or scanning for advertising frames transmitted by otherelectronic devices.

Within electronic device 200, processing subsystem 210, memory subsystem212, networking subsystem 214, display subsystem 226, measurementsubsystem 230, and user-interaction subsystem 232 are coupled togetherusing bus 228 that facilitates data transfer between these components.Bus 228 may include an electrical, optical, and/or electro-opticalconnection that the subsystems can use to communicate commands and dataamong one another. Although only one bus 228 is shown for clarity,different embodiments can include a different number or configuration ofelectrical, optical, and/or electro-optical connections among thesubsystems.

In some embodiments, electronic device 200 includes display subsystem226 for displaying information on a display, which may include a displaydriver and the display, such as a liquid-crystal display, a multi-touchtouchscreen, etc. Display subsystem 226 may be controlled by processingsubsystem 210 to display information to a user (e.g., informationrelating to incoming, outgoing, or an active communication session). Insome embodiments, display subsystem 226 can be configured to generatedisplay information for non-native displays (e.g., displays associatedwith other devices).

Electronic device 200 can also include a measurement subsystem 230 withone or more sensors that allows electronic device 200 to perform one ormore type of measurements. For example, the one or more sensors mayinclude: one or more compasses, one or more accelerometers, one or moregyroscopes, one or more microphones or acoustic transducers, one or moreenvironmental sensors (such as a temperature sensor and/or analtimeter), one or more light sensors (such as an ambient light sensor),one or more touch sensors (such as a touchscreen), one or more biometricsensors (such as a fingerprint sensor), etc. Note that the one or moresensors may include physical sensors in electronic device 200 and/or avirtual sensor (such as a sensor implemented, at least in part, insoftware). In some embodiments, at least some of the one or more sensorsdetermine sensor data based on information received from a remoteelectronic device.

Moreover, electronic device 200 may include a user-interaction subsystem232. For example, user-interaction subsystem 232 can include a varietyof user-input devices, such as: a button, keypad, dial, touchscreen,audio-input interface, visual/image-capture-input interface, input inthe form of sensor data, etc. Alternatively or additionally,user-interaction subsystem 232 may include a variety of user-outputdevices, such as: one or more speakers (which may provide a directionalacoustic array), a haptic transducer, etc.

Electronic device 200 can be (or can be included in) any electronicdevice with at least one network interface. For example, electronicdevice 200 may include: a cellular telephone or a smartphone, a tabletcomputer, a laptop computer, a notebook computer, a personal or desktopcomputer, a netbook computer, a computing device, a shared computingdevice (such as a printer), a media player device, an electronic bookdevice, a smart watch, a wearable computing device, a wearable device, aportable computing device, a consumer-electronic device, an accesspoint, a router, a switch, communication equipment or a communicationdevice, test equipment, an appliance, an entertainment device, atelevision, a display, a radio receiver, a set-top box, an audio device,a projector, a medical device (such as an automated externaldefibrillator), a security device, an alarm, a monitoring device (e.g.,a smoke detector or a carbon-monoxide detector), an environmentalcontrol, a thermostat, a light switch, an accessory, a keyboard, amouse, a speaker, a printer, a home-automation device, a vehicle, anelectronic lock, as well as any other type of electronic computingdevice having wireless communication capability that can includecommunication via one or more wireless communication protocols.

Although specific components are used to describe electronic device 200,in alternative embodiments, different components and/or subsystems maybe present in electronic device 200. For example, electronic device 200may include one or more additional processing subsystems, memorysubsystems, networking subsystems, and/or display subsystems.Additionally, one or more of the subsystems may not be present inelectronic device 200. Moreover, in some embodiments, electronic device200 may include one or more additional subsystems that are not shown inFIG. 2 . Also, although separate subsystems are shown in FIG. 2 , insome embodiments some or all of a given subsystem or component can beintegrated into one or more of the other subsystems or component(s) inelectronic device 200. For example, in some embodiments program module222 is included in operating system 224 and/or control logic 216 isincluded in interface circuit 218.

Moreover, the circuits and components in electronic device 200 may beimplemented using any combination of analog and/or digital circuitry,including: bipolar, PMOS and/or NMOS gates or transistors. Furthermore,signals in these embodiments may include digital signals that haveapproximately discrete values and/or analog signals that have continuousvalues. Additionally, components and circuits may be single-ended ordifferential, and power supplies may be unipolar or bipolar.

An integrated circuit (which is sometimes referred to as a‘communication circuit’) may implement some or all of the functionalityof networking subsystem 214. This integrated circuit may includehardware and/or software mechanisms that are used for transmittingwireless signals from electronic device 200 and receiving signals atelectronic device 200 from other electronic devices. Aside from themechanisms herein described, radios are generally known in the art andhence are not described in detail. In general, networking subsystem 214and/or the integrated circuit can include any number of radios. Notethat the radios in multiple-radio embodiments function in a similar wayto the described single-radio embodiments.

In some embodiments, networking subsystem 214 and/or the integratedcircuit include a configuration mechanism (such as one or more hardwareand/or software mechanisms) that configures the radio(s) to transmitand/or receive on a given communication channel (e.g., a given carrierfrequency). For example, in some embodiments, the configurationmechanism can be used to switch the radio from monitoring and/ortransmitting on a given communication channel to monitoring and/ortransmitting on a different communication channel. (Note that‘monitoring’ as used herein includes receiving signals from otherelectronic devices and possibly performing one or more processingoperations on the received signals, e.g., performing wireless ranging,etc.)

In some embodiments, an output of a process for designing the integratedcircuit, or a portion of the integrated circuit, which includes one ormore of the circuits described herein may be a computer-readable mediumsuch as, e.g., a programmable memory, a magnetic tape, or an optical ormagnetic disk. The computer-readable medium may be encoded with datastructures or other information describing circuitry that may bephysically instantiated as the integrated circuit or the portion of theintegrated circuit. Although various formats may be used for suchencoding, these data structures are commonly written in: CaltechIntermediate Format (CIF), Calma GDS II Stream Format (GDSII) orElectronic Design Interchange Format (EDIF). Those of skill in the artof integrated circuit design can develop such data structures fromschematic diagrams of the type detailed above and the correspondingdescriptions and encode the data structures on the computer-readablemedium. Those of skill in the art of integrated circuit fabrication canuse such encoded data to fabricate integrated circuits that include oneor more of the circuits described herein.

The user-interface technique may be used in a variety of networkinterfaces. Furthermore, while some of the operations in the precedingembodiments were implemented in hardware or software, in general theoperations in the preceding embodiments can be implemented in a widevariety of configurations and architectures. Therefore, some or all ofthe operations in the preceding embodiments may be performed inhardware, in software or both. For example, at least some of theoperations in the user-interface technique may be implemented usingprogram module 222, operating system 224 (such as a driver for interfacecircuit 218) or in firmware in interface circuit 218. Alternatively oradditionally, at least some of the operations in the user-interfacetechnique may be implemented in a physical layer, such as hardware ininterface circuit 218. In an exemplary embodiment, the user-interfacetechnique is implemented, at least in part, in a MAC layer in interfacecircuit 218.

In an exemplary embodiment, the user-interface technique is used toprovide directional control. In particular, when the position andorientation of a wireless communication device relative to an object aredetermined based on one or more measurements, one or more gestures inputon or performed with the wireless communication device may beinterpreted as a command for the object. As described further below,wireless ranging and sensor-data (from a gyroscope, an accelerometer, acompass, etc.) inputs may be used to select, pair with, and/or commandthe object.

FIG. 3 provides a drawing 300 illustrating an example of control of anobject using gestures based at least in part on wireless ranging. Inparticular, during the user-interface technique, a user may use wirelesscommunication device 110 to control a physical device, e.g., physicaldevice 112, or to generate control input for a physical device usinginteraction with a virtual representation of an object, e.g., virtualrepresentation 114 by performing one or more gestures or motions, e.g.,in two or three dimensions. Wireless communication device 110 isillustrated generating gesture input with respect to object 310, whichcan be an actual device or a virtual representation. For example, a usermay use their cellular telephone to control the volume or the selectedchannel of a television, or to turn the room lights on or off byperforming a gesture, such as by moving wireless communication device110 up and then down (or vice versa). Alternatively, the user maycontrol a virtual representation of a light switch at a predefinedphysical location (such as physical location 116 in FIG. 1 ) in a roomor an environment 108 in FIG. 1 (i.e., there may not be a physical lightswitch at the predefined physical location). Note the predefinedphysical location (and, more generally, a predefined physical locationin three-dimensional space) may have been specified by the user usingwireless communication device 110 during a training process, such as bybringing wireless communication device 110 proximate to or touching thephysical location and activating a virtual command icon in a userinterface displayed on a touch-sensitive display on wirelesscommunication device 110, or by touching wireless communication device110 to the physical location and verbally stating a command or aninstruction that is monitored by wireless communication device 110 andinterpreted using a speech-recognition technique.

During the user-interface technique, the user may specify (or otherwiseindicate) object 310 using an ‘intent gesture’ 312, such as by extendingwireless communication device 110 toward or in the general direction ofobject 310 (such as, e.g., within 5° of object 310). Then, the user mayperform an ‘action gesture’ 314, e.g., by moving wireless communicationdevice 110 relative to object 310. In one example, wirelesscommunication device 110 can be moved up or down relative to object 310,e.g., to signal an increase or decrease in a value, respectively. Othergestures also may be used, such as by providing lateral and/orrotational movement. Moreover, a gesture can be simple (e.g., involvingone movement, such as a vertical or horizontal movement) or complex(e.g., involving two or more movements, such as a combination ofhorizontal, vertical, lateral, and/or rotational movements). Theseoperations may be performed while the user is pointing wirelesscommunication device 110 toward or in the general direction of object310.

In order to detect intent gesture 312, wireless communication device 110may measure its orientation relative to the position or location ofobject 310. For example, object 310 may be identified based on a genericor a unique identifier that is communicated by object 310, and therelative position may be determined based on the wireless ranging (e.g.,using wireless signals 122), the metric (such as the RSSI), and/or theorientation. Moreover, to detect action gesture 314, wirelesscommunication device 110 may measure the movement, acceleration, and/ordistance 316, e.g., using sensor input and/or wireless ranging. Notethat the wireless ranging may be performed at or in one or more bands offrequencies, such as at or in: a 2 GHz wireless band, a 5 GHz wirelessband, an ISM band, a 60 GHz wireless band, an ultra-wide band, etc.

In addition, ambient sound may be used by wireless communication device110 to determine whether object 310 is in the same environment 108 (FIG.1 ) as wireless communication device 110, such as the same room (asopposed to an adjacent or neighboring room). For example, the ambientsound recorded by object 310, which is shared with wirelesscommunication device 110 via a wireless communication channel (such as aWLAN), may be compared with the ambient sound measured by wirelesscommunication device 110. If these are the same, wireless communicationdevice 110 and object 310 may be in the same room. Alternatively oradditionally, wireless communication device 110 may output one or moreacoustic chirps or sound(s) (which may be at or in a band of frequenciesthat are outside of the range of frequencies a human can hear) that canbe detected by object 310, e.g., if there is no intervening physicalbarrier.

More generally, one or more of the measurements performed by wirelesscommunication device 110 may be used to disambiguate one or morepotential target devices in environment 108 (FIG. 1 ), such as object310, from one or more other devices, such as physical device 118. Forexample, object 310 facing the user of wireless communication device 110may have the same orientation, a different height, and/or the same or adifferent RSSI as another object in the environment. In conjunction withdistance 316, the one or more measurements may be used to identifyobject 310 from all of the available devices. Furthermore, as notedpreviously, a potential target device may be excluded based on theambient sound. For example, ambient sound may indicate that a device islocated in a pocket or purse and, thus, is less likely to be the targetof an intent gesture 312 for pairing or sharing of information.

In some implementations, an altimeter can be used to exclude one or morepotential target devices, e.g., that are below a table or are located ina pocket/bag, when they are not likely intended to be designated by anintent gesture 312. In another example, one or more ultrasonic chirps or60 GHz transmissions can be used to exclude potential target deviceswithin primary radio range or wireless proximity of wirelesscommunication device 110, but that are outside of a relevant physicalspace or physical proximity (e.g., the excluded potential target devicesmay be located in a different room or stored inside of a container).

When two or more potential target devices cannot be separated in one ormore of these ways, then the user of wireless communication device 110may be asked to choose among a set of potential target devices that wereidentified. For example, a user interface with a list of potentialtarget devices may be displayed on a display in wireless communicationdevice 110. The user can then select the intended target, e.g., of theintent gesture 312, from the interface and provide a desired actiongesture 314 or other such control input.

While the preceding discussion included illustrative examples of intentgesture 312 and action gesture 314, in general these gestures mayinclude a wide variety of motions and rotations in two or threedimensions, such as: linear motion in one or more directions, rotationabout one or more axes, motion along an arc, virtual tapping,figure-based gestures (e.g., a figure-8 gesture), a gesture in the shapeof a letter or a number, combinations of linear motion(s) androtation(s), etc. More generally, action gesture 314 may have anintuitive correspondence to the type of change or the desired controloperation. For example, as shown in FIG. 4 , which provides a drawing400 illustrating an example of control of an object using gestures, anup and/or down action gesture 410 may be used to increase (or decrease)a value or setting, such as the temperature setting of a heater orthermostat, or the position of an object (such as a window share orscreen). Similarly, as shown in FIG. 5 , which provides a drawing 500illustrating an example of control of an object using gestures, actiongesture 510 may involve a left and/or right rotation that corresponds tochanging a value or setting, such as the volume of an entertainmentdevice.

In one example, a user may point their cellular telephone at atelevision. Through wireless ranging, the cellular telephone maydetermine that the television is within range (i.e., is close enough tobe the intended target device). Accelerometer and/or gyroscope input mayprovide a further indication of relative motion that can be interpretedas an intent gesture 312, indicating that wireless communication device110 is being used to command (e.g., the action gesture) the television,e.g., object 310. Further, one or more action gestures (e.g., any of314, 410, and/or 510) can be provided to indicate one or more commandsto control the television. For example, the user can twist the cellulartelephone to the right to increase the volume of the television or tochange the channel.

The user-interface technique may also be used to intuitively pair thewireless communication device 110 with another electronic device so thatthe paired electronic devices can subsequently communicate with eachother. For example, by performing the intent gesture 312 and the actiongesture (314, 410, and/or 510), a user may be able to remotely indicatea desire to exchange or share data wirelessly with the physical device.When these operations are performed, the wireless communication device110 and/or the physical device may display a message in a userinterface, requesting confirmation and/or authorization for the pairing.In addition, the other physical device may agree to the pairing for aspecified time interval (e.g., 10 min or an hour), after which thepairing may be disabled. In this way, the user of wireless communicationdevice 110 may maintain control of the pairing process, which also mayhelp ensure that the pairing is secure, such as by confirming that thepaired physical device is in the same environment (such as the sameroom). By providing directional intent and action gestures, wirelesscommunication device 110 may avoid pairing with a device that is tryingto electronically spoof or impersonate the object 310, with whichpairing is intended. Similarly, the directional intent and actiongestures can avoid accidental pairing with a device other than object310 that is proximate, but at a different location or elevation withinthe environment. This approach may also help to reduce the complexity ofsharing information and/or pairing. For example, passwords may not beneeded during the pairing process. In some embodiments, after the intentgesture and the action gesture(s), the wireless communication device andthe physical device can exchange encryption keys.

In another example, the wireless communication device 110 can be pointedin the direction of an object with which there is intent to pair or forwhich there is intent to issue a command. The camera of wirelesscommunication device 110 may capture an image of all objects in thefield of view and can present a selectable, on-screen interface showingone or more candidate devices. The user can then select the device (andoptionally, e.g., protocol) with which to pair.

In still another example, wireless communication device 110 can bepointed at object 310 to claim a resource or to obtain information. Forinstance, wireless communication device 110 can be pointed (e.g., usingintent gesture 312 with or without an action gesture 314, 410, and/or510) at a cab (e.g., object 310) to hail it. Further, wirelesscommunication device 110 can be pointed (e.g., using intent gesture 312with or without an action gesture 314, 410, and/or 510) at a restaurant(or an object 310 associated with the restaurant) to obtain ratings orto make a reservation. Similarly, wireless communication device 110 caninteract with (e.g., set-up or print to) a printer by pointing at it.For example, an intent gesture 312 and an action gesture 314, 410,and/or 510 can be provided at the wireless communication device 110 tosend information to a printer (e.g., to initiate a print job) or toinitiate control of one or more settings/configurations.

Wireless ranging and directionality also may be used to obtaininformation from one or more sensors. For example, wirelesscommunication device 110 can be pointed toward a sensor (e.g., athermostat or a tire pressure gauge) to obtain a reading. In someimplementations, an intent gesture 312 with or without an action gesture314, 410, and/or 510 can be used to initiate communication of the sensordata. Similarly, one or more intent and/or action gestures can beprovided at wireless communication device 110 to retrieve informationfrom a server, such as by interacting with an item or a display in astore to check an attribute, such as price or availability information.

Moreover, identification of object 310, e.g., by wireless communicationdevice 110, may be facilitated by an identifier (generic or unique) thatis selectively transmitted by object 310. For example, the identifiermay be a persistent, unique identifier that is associated with aparticular electronic device (e.g., object 310). Wireless communicationdevice 110 can store the persistent, unique identifier (e.g. locally orin the cloud) and can associate the identifier with a person, object, orlocation (e.g., in contact information). When the identifier is detectedin a received wireless communication, the wireless communication device110 can provide an indication that the known electronic device (e.g.,object 310) is proximate. Alternatively, the identifier may be transientor temporary, such as an identifier assigned by an application executingon wireless communication device 110 or object 312. For example, aride-share application may temporarily assign an identifier to wirelesscommunication device 110, which it can then broadcast. Further, theride-share application can provide the transient identifier to a devicecorresponding to an assigned driver to facilitate pick-up. Similarly, anidentifier (persistent or transient) corresponding to the driver'sdevice can be provided to wireless communication device 110 to verifythe driver. A transient identifier may expire after a predeterminedamount of time, such as an hour, or upon completion of a transaction.Any number of persistent and transient identifiers can be associatedwith a device or an object, e.g., to facilitate identification across avariety of scenarios.

Furthermore, an assigned identifier may be varied over time to protectthe user's privacy and to provide security. In particular, an identifiermay be assigned by an optional server 126 in FIG. 1 (which may beaccessed via network 124 in FIG. 1 , such as a wired or a wirelessnetwork) and may be updated periodically (such as after a time interval,e.g., 5 min or 1 day, has elapsed). Additionally or alternatively, anidentifier may be assigned as a one-time identifier, e.g., for aparticular transaction. In some embodiments, the identifier can be ananonymous identifier that can be persistently assigned to a device orobject, such as object 310. The anonymous identifier can then only beresolved by other devices with which the anonymous identifier and anitem of identification information have been shared.

In order to reduce power consumption, wireless communication device 110and/or object 312 may only transmit one or more identifier(s)periodically or upon an event, such as detection of an identifierassociated with a known contact. Additionally or alternatively, wirelesscommunication device 110 and/or object 312 may only attempt to receiveone or more identifier(s) when one or more conditions are met, such asbeing in proximity of another device (known or unknown), being engagedin a transaction involving another device, being in a particular are(e.g., within a geo-fenced region), in response to user input, and/orbased on an event (e.g., an appointment or meeting). For example, aBluetooth low energy (BLE) receiver may periodically scan for anadvertisement (or other message) that includes an identifier from, e.g.,object 312. The scan rate can vary based on one or more factors, such asmotion, time, and/or available power. In response to detecting a knownidentifier, wireless communication device 110 may activate one or moreother communication interfaces and may enable wireless ranging, e.g., sothat wireless communication device 110 can receive a message from object312 and/or determine a direction and/or distance 316 to object 312.

Moreover, as discussed above, the user-interface technique mayfacilitate the use of virtual objects. These embodiments may includedefining virtual objects and controlling them through wirelessranging/directional control input. In some embodiments, interaction witha virtual object, e.g., virtual representation 114, can be based on theorientation of the wireless communication device 110 and a receivedintent gesture, without performing wireless ranging.

For example, a virtual object or a virtual representation can beestablished in a particular physical location, such as a virtual lightswitch or a virtual thermostat by a door. The virtual object can bephysically represented by a visual item (e.g., sticker) or can be purelyvirtual, corresponding only to one or more coordinates. The virtualobject may also be defined by an associated physical device. Forexample, a projector can define a virtual control, e.g., on a wall, andcan advertise the virtual control's position, allowing other electronicdevices to manipulate the virtual control to effectuate real input tothe projector.

As noted previously, in embodiments with virtual objects a device (e.g.,wireless communication device 110) may define a virtual device in aparticular location, e.g., a virtual control corresponding to aparticular location on a wall. Note that the virtual device can bedefined as part of a larger system. For example, a facility may define avirtual thermostat adjacent to every entry door (e.g., at a lightswitch). Alternatively, the virtual device may be defined by aparticular device, for example, a projector may define a control on awall or other surface and may wirelessly advertise the position of thecontrol, thereby allowing other electronic devices to manipulate thevirtual control as though it were real.

In this way, a wireless communication device 110 (e.g., a cellulartelephone or wearable) may be able to function as a control device,e.g., a dynamic remote control. Further, the wireless communicationdevice 110 may be able to provide control input to a wide-variety ofobjects, e.g., object 310, through gestures, without been configured tohave specific input mechanisms. This may allow the display of wirelesscommunication device 110 to serve as a proxy touch screen for a largermonitor, and sensors/inputs such as the compass, gyroscope,accelerometer, altimeter, touch input and/or microphone can be used tocontrol inputs and/or outputs associated with the monitor. In someembodiments, wireless communication device 110 can be used as a secondscreen corresponding to a primary display (note that it may or may notmirror the primary display). Then, gesture-based (including touch-based)controls can be provided to control the primary display (e.g., pinch,zoom, scroll, swipe etc.).

Furthermore, gesture-based control input, e.g., to wirelesscommunication device 110, may be used to perform functions such as thetransfer of content between electronic devices. For example, one or moregesture-based control inputs to wireless communication device 110 can beused to send (e.g., “throw”) content to a second electronic device, topull content from a second electronic device, to take control of thesecond electronic device (e.g., take ownership of a computer and have itreplicate your content/settings), etc. Thus, one or more gesture-basedinputs (e.g., flip, push, rub, swipe, or tap) can be used to indicate atransfer from (or to) wireless communication device 110 to (or from)another device, such as a larger display or a computer. Similarly,gesture-based input to wireless communication device 110 can be used tomove something from one device to another, using wireless communicationdevice 110 as a data carrier.

In some implementations, a virtual object may encompass constructs suchas a virtual queue (or line), e.g., for service. For example, a virtualqueue can be defined in association with an entity or resource (e.g., aticket booth or store) and electronic devices in proximity can registerin the virtual queue by transmitting an associated identifier (e.g., aunique, transient identifier) to a host device maintaining the virtualqueue. The virtual queue position for a device (e.g., wirelesscommunication device 110) can be maintained while the device remains inproximity of the physical location of the virtual queue or until thequeue is processed. Further, the position in the virtual queue also canbe presented to the registered device and can be updated. The hostdevice can notify a registered device when its turn (e.g., at the ticketbooth) arrives or approaches.

In addition to cost savings, the use of virtual representations orvirtual objects for corresponding physical devices also may allow theenvironment to be more aesthetically pleasing by replacing lessattractive physical devices (such as light switches) with an attractiveplaceholder (such as a ceramic object) or by removing physical devicesaltogether. This approach also allows a virtual representation to bemoved, at will, by simply re-defining the location associated with thevirtual representation. Then, when the user performs the intent gestureand the action gesture when pointing their electronic device, e.g.,wireless communication device 110, toward the new location, anassociated operation(s) may be performed.

In some embodiments, the gesture used to control the associated physicaldevice may be implicit. In such embodiments, proximity determined bywireless ranging may be used to engage a controllable device, withoutassociated intent and action gestures. For example, a light may turn on(and then off) as a user walks by a predefined location associated witha virtual representation or a virtual object. Similarly, a door lock mayunlock as a user device approaches a door, a traffic light may change togreen (if conditions permit) when the user device arrives at anintersection, etc. Thus, the disclosed techniques may be used forproximity-based control.

Moreover, the disclosed techniques may also be used to providecooperative situational awareness, such as when an electronic deviceahead of the direction of travel experiences an event, e.g., undergoinga sudden deceleration. The electronic device can notify other electronicdevices, e.g., through a broadcast message, of a potentially dangeroussituation.

In some embodiments, the disclosed techniques are facilitated by one ormore optional tags, such as optional tag 128 (FIG. 1 ) in environment108 (FIG. 1 ). For example, a tag may be a low-cost system-on-a-chipthat harvests power from radio-frequency energy and/or has an associatedpower supply. The tag 128 can output one or more signals, e.g., inresponse to a received command. For example, the tag 128 can output RFsignals and/or acoustic signals (such as “chirps”).

In some embodiments, a tag 128 in environment 108 (FIG. 1 ) harvestspower over the air (e.g., it may receive power in the form of signals at24 GHz) and communicates through a brief signal or chirp (e.g., byemitting a signal at 60 GHz). This communication may have a range up to,e.g., 100 ft., and may be directional. In some implementations, the 24GHz power signal may have an identifier encoded in the signal. In suchimplementations, if the tag does not recognize the identifier, it maynot respond. The time duration for a tag to respond to a start pulse(e.g., once it has indicated it is powered and operational) may beknown. Note that a wireless communication device may measure the phaseand angle of arrival to determine the direction, and may use the time ofarrival or the time of flight to determine the distance between thewireless communication device and the tag. Accordingly, the tag 128 maybe used to generate or to supplement wireless ranging computations.

FIG. 6 is a flow diagram illustrating an example of a method 600 forcontrolling an object using a wireless communication device, such aswireless communication device 110 in FIG. 1 . During operation, thewireless communication device identifies a received intent gesture(operation 610) indicating intent to control an object located proximateto the wireless communication device. Then, the wireless communicationdevice determines the object associated with the intent gesture(operation 612). For example, the wireless communication device useswireless ranging to determine an object within range that also islocated in a direction and/or elevation corresponding to a positionand/or orientation of the wireless communication device, e.g., based atleast in part on the intent gesture. Further, the wireless communicationdevice accesses sensor data associated with one or more sensors of thewireless communication device (operation 614) and interprets the sensordata to determine a command value (operation 616). For example, thewireless communication device compares the sensor data to one or moreaction gestures (e.g., based on action gesture profiles stored on thewireless communication device) to identify one or more intendedcommands. The wireless communication device then generates and transmitsa command value, e.g., using one or more messages, to control the object(operation 618). In some implementations, the one or more messages canbe transmitted to the object based on an identifier advertised by, orotherwise associated with, the object.

In some embodiments of method 600, there may be additional, different,and/or fewer operations. Furthermore, the order of the operations may bechanged, and/or two or more operations may be combined into a singleoperation or performed in parallel.

Embodiments of the user-interface technique are further illustrated inFIG. 7 , which presents a drawing 700 illustrating communication amongwireless communication device 110 and physical (or electronic) device112 (which is used as an illustrative example). A user may perform anintent gesture 708, e.g., by pointing or orienting 710 wirelesscommunication device 110 toward physical device 112. In someimplementations, the intent gesture 708 may include both an orientationcomponent and a motion component, such as a motion advancing wirelesscommunication device 110 toward physical device 112. Processor 714 inwireless communication device 110 may identify the intent gesture 708,e.g., based on orientation 710 and any other associated motion, asmeasured by one or more sensors 712 (e.g., a gyroscope, accelerometer,and/or compass). The intent gesture 708 may be interpreted as userintent to control physical device 112.

Interface circuit 718 in wireless communication device 110 receives apacket 720 (or frame) transmitted by interface circuit 716 in physicaldevice 112. Packet 720 includes an identifier 722 associated withphysical device 112. Further, packet 720 can include informationspecifying a transmission time of packet 720. In conjunction with adetermined arrival time of packet 720 (based on the speed of propagationof the wireless signals that conveyed packet 720), interface circuit 718(or processor 714) may determine a distance 724 to physical device 112.Additionally or alternatively, wireless communication device 110 cancommunicate one or more other packets with physical device 112 toperform a wireless ranging operation. Based on identifier 722 and/ordistance 724, processor 714 may identify physical device 112.

Further, an action gesture may be performed at wireless communicationdevice 110 corresponding to a command or a command value. One or more ofsensors 712 measure sensor data 726 during the action gesture and maystore the measured sensor data in memory 728. Processor 714 may accesssensor data 726 (e.g., by receiving it from the sensors 712 and/orretrieving it from memory 728), and interpret the sensor data 726 todetermine a command value 730 corresponding to the action gesture.Further, interface circuit 718 may transmit a packet 732 including thecommand value 730 to interface circuit 716. Upon receiving the packet732, physical device 112 can implement the command value 730.

Representative Embodiments

In some embodiments, a wireless-communication-device-implemented methodfor controlling an object includes: (i) identifying, by the wirelesscommunication device, an intent gesture indicating an intent to controlthe object, where the object is located proximate to the wirelesscommunication device; (ii) determining, by the wireless communicationdevice, the object associated with the intent gesture based at least inpart on a wireless ranging operation; (iii) accessing sensor datagenerated by one or more sensors associated with the wirelesscommunication device; (iv) interpreting the sensor data to determine acommand value; and (v) transmitting the command value to control theobject.

In some embodiments, the object includes a physical object. In someembodiments, the physical object includes a computing device, a display,a printer, a communications device, an audio device, an appliance, awearable device, a home automation device, an environmental control, oran accessory. In some embodiments, the object includes a virtual object.In some embodiments, the virtual object includes a proxy for a physicalobject at a different location. In some embodiments, the wirelessranging operation includes using a wireless transmission characteristicto determine a distance between the wireless communication device andthe object. In some embodiments, the accessed sensor data includes anorientation value corresponding to the wireless communication device,the orientation value being determined using at least one of: a compass,an accelerometer, or a gyroscope. In some embodiments, the command valueincludes a command to pair the wireless communication device with theobject. In some embodiments, the command value includes a command tochange an operational setting of the object. In some embodiments, thecommand value includes a command to execute a function. In someembodiments, the method further includes using an ultrasonic chirp todetermine proximity of the object. In some embodiments, the methodfurther includes receiving from the object, responsive to the commandvalue, a response including a sensor value. In some embodiments, thecommand value includes a request for access to a resource.

In some embodiments, a wireless communication device includes: (i) aninterface circuit configured to use at least one wireless communicationprotocol to communicate wirelessly with an object; (ii) one or moresensors configured to measure sensor data; (iii) a processor,communicatively coupled to the interface circuit and the one or moresensors, configured to execute a program module; and (iv) a memory,communicatively coupled to the processor, configured to store theprogram module, the program module includes instructions for:identifying, by the wireless communication device, an intent gestureindicating an intent to control the object, where the object is locatedproximate to the wireless communication device; determining, based atleast in part on wireless ranging, the object associated with the intentgesture; accessing sensor data generated by the one or more sensors;interpreting the accessed sensor data to identify an action gesture; andtransmitting, using the interface circuit, a command value correspondingto the action gesture to control the object.

In some embodiments, the action gesture includes a complex gesture. Insome embodiments, the determining the object is further based at leastin part on receiving an identifier associated with the object. In someembodiments, the program module further includes instructions forinitiating a pairing operation with the object.

In some embodiments, a computer-program product for use in conjunctionwith a wireless communication device includes a non-transitorycomputer-readable storage medium and a computer-program mechanismembedded therein to control an object, the computer-program mechanismincluding: (i) instructions for identifying, by the wirelesscommunication device, an intent gesture indicating an intent to controlthe object, where the object is located proximate to the wirelesscommunication device; (ii) instructions for determining, by the wirelesscommunication device, the object associated with the intent gesturebased at least in part on wireless ranging; (iii) instructions foraccessing, by the wireless communication device, sensor data generatedby one or more sensors associated with the wireless communicationdevice; (iv) instructions for interpreting the accessed sensor data todetermine an action gesture; and (v) instructions for transmitting acommand corresponding to the action gesture to control the object.

In some embodiments, the instructions for determining the object furtherinclude instructions for determining an orientation of the wirelesscommunication device, based at least in part on sensor data generated bythe one or more sensors associated with the wireless communicationdevice. In some embodiments, the instructions for determining the objectfurther include instructions for ignoring another object based at leastin part on the orientation of the wireless communication device and thewireless ranging.

In the preceding description, we refer to ‘some embodiments.’ Note that‘some embodiments’ describes a subset of all of the possibleembodiments, but does not always specify the same subset of embodiments.

The foregoing description is intended to enable any person skilled inthe art to make and use the disclosure, and is provided in the contextof a particular application and its requirements. Moreover, theforegoing descriptions of embodiments of the present disclosure havebeen presented for purposes of illustration and description only. Theyare not intended to be exhaustive or to limit the present disclosure tothe forms disclosed. Accordingly, many modifications and variations willbe apparent to practitioners skilled in the art, and the generalprinciples defined herein may be applied to other embodiments andapplications without departing from the spirit and scope of the presentdisclosure. Additionally, the discussion of the preceding embodiments isnot intended to limit the present disclosure. Thus, the presentdisclosure is not intended to be limited to the embodiments shown, butis to be accorded the widest scope consistent with the principles andfeatures disclosed herein.

1. (canceled)
 2. A method comprising: by a wireless device: determiningan intent to control an object within wireless range of the wirelessdevice; identifying the object to be controlled based at least in parton an anonymous identifier received from the object during a wirelessranging operation, the anonymous identifier distinguishing the objectfrom a second object; determining a command to control the objectidentified by the wireless ranging operation; and transmitting a commandvalue corresponding to the command to control the object.
 3. The methodof claim 2, wherein determining the intent to control the objectcomprises interpreting sensor data of the wireless device to identify anintent gesture.
 4. The method of claim 3, wherein interpreting thesensor data includes identifying an orientation of the wireless device.5. The method of claim 3, wherein interpreting the sensor data includesidentifying a movement of the wireless device.
 6. The method of claim 5,wherein the movement of the wireless device comprises a predefinedintent gesture motion.
 7. The method of claim 6, wherein the predefinedintent gesture motion comprises a forward extension of the wirelessdevice toward the object to be controlled.
 8. The method of claim 2,wherein determining the command to control the object comprisesinterpreting sensor data to identify an action gesture.
 9. The method ofclaim 8, wherein the action gesture comprises movement of the wirelessdevice in at least two dimensions.
 10. The method of claim 2, whereinthe object comprises a physical object.
 11. The method of claim 2,wherein the object comprises a virtual object that represents a physicalobject in a different location.
 12. The method of claim 2, furthercomprising: by the wireless device: determining whether the object is inthe same environment as the wireless device.
 13. The method of claim 12,wherein the determining whether the object is in the same environment asthe wireless device comprises comparing ambient sound measured by thewireless device to ambient sound captured by the object.
 14. The methodof claim 12, wherein the determining whether the object is in the sameenvironment as the wireless device comprises transmitting one or moreultrasonic acoustic chirps.
 15. A wireless device comprising: wirelesscircuitry; and a processor communicatively coupled to the wirelesscircuitry and to a memory storing instructions that, when executed bythe processor, configure the wireless device to perform actionsincluding: determining an intent to control an object within wirelessrange of the wireless device; identifying the object to be controlledbased at least in part on an anonymous identifier received from theobject during a wireless ranging operation, the anonymous identifierdistinguishing the object from a second object; determining a command tocontrol the object identified by the wireless ranging operation; andtransmitting a command value corresponding to the command to control theobject.
 16. The wireless device of claim 15, further comprising asensor, wherein determining the intent to control the object comprisesinterpreting sensor data to identify a predefined intent gesturecomprising movement of the wireless device toward the object to becontrolled.
 17. The wireless device of claim 15, further comprising asensor, wherein determining the command to control the object comprisesinterpreting sensor data to identify a predefined action gesturecomprising movement of the wireless device in at least two dimensions.18. The wireless device of claim 15, wherein the actions further includedetermining whether the object is in the same environment as thewireless device based on analysis of ambient acoustic data.
 19. Thewireless device of claim 15, wherein the object comprises a virtualobject associated with a physical object in a different location.
 20. Acomputer-program product for use in conjunction with a wireless device,the computer-program product comprising a non-transitorycomputer-readable storage medium and a computer-program mechanismembedded therein to control an object, the computer-program mechanismincluding: instructions for determining an intent to control an objectwithin wireless range of the wireless device; instructions foridentifying the object to be controlled based at least in part on ananonymous identifier received from the object during a wireless rangingoperation, the anonymous identifier distinguishing the object from asecond object; instructions for determining a command to control theobject identified by the wireless ranging operation; and instructionsfor transmitting a command value corresponding to the command to controlthe object.
 21. The computer-program product of claim 20, wherein theobject comprises a virtual object that represents a physical object in adifferent location.